Waning effectiveness of mRNA COVID-19 vaccines against inpatient and emergency department encounters

In the United States, most real-world estimates of COVID-19 vaccine effectiveness are based on data drawn from large health systems or sentinel populations. More data is needed to understand how the benefits of vaccination may vary across US populations with disparate risk profiles and policy contexts. We aimed to provide estimates of mRNA COVID-19 vaccine effectiveness against moderate and severe outcomes of COVID-19 based on state population-level data sources. Using statewide integrated administrative and clinical data and a test-negative case-control study design, we assessed mRNA COVID-19 vaccine effectiveness against SARS-CoV-2-related hospitalizations and emergency department visits among adults in South Carolina. We presented estimates of vaccine effectiveness at discrete time intervals for adults who received one, two or three doses of mRNA COVID-19 vaccine compared to adults who were unvaccinated. We also evaluated changes in vaccine effectiveness over time (waning) for the overall sample and in subgroups defined by age. We showed that while two doses of mRNA COVID-19 vaccine were initially highly effective, vaccine effectiveness waned as time elapsed since the second dose. Compared to protection against hospitalizations, protection against emergency department visits was found to wane more sharply. In all cases, a third dose of mRNA COVID-19 vaccine conferred significant gains in protection relative to waning protection after two doses. Further, over more than 120 days of follow-up, the data revealed relatively limited waning of vaccine effectiveness after a third dose of mRNA COVID-19 vaccine.


Introduction
Messenger RNA (mRNA) vaccines are effective in preventing COVID-19 infection and virusrelated complications [1,2].Extended trials and real-world effectiveness studies have shown,

Data sources
This study used integrated data from two statewide databases with mandated reporting overseen by the SC Department of Health and Environmental Control-COVID-19 testing data and the State's immunization registry-and a statewide all-payer claims database (hospital inpatient, outpatient, and ED).The SC Office of Revenue and Fiscal Affairs carried out the necessary data linkages and provided the research team with a de-identified dataset.The University of South Carolina Institutional Review Board determined that this study met the Not Human Subject criteria set forth by the Code of Federal Regulations (45 CFR 46).Since this study involved secondary analysis of existing data sources and no more than minimal risk, consent was waived for the study.The data were accessed on February 14, 2023.

Study design
We used a retrospective test-negative case-control design to estimate mRNA VE against SARS-CoV-2-related hospitalizations and ED visits.The test-negative design is a widely applied statistical approach for monitoring VE in observational studies [24].In order to mitigate potential biases resulting from differential care-seeking behaviors, we restricted the sample to patients who presented with COVID-like illness (CLI) [9,[25][26][27].VE was estimated by comparing the odds of prior COVID-19 vaccination among symptomatic patients who tested positive for SARS-CoV-2 (cases) with the odds among symptomatic patients who had a negative test result (controls).We estimated VE by number of vaccine doses and time elapsed since the last vaccine dose.Separate analyses were conducted for inpatients and ED patients.

Study population and setting
The study population included SC adults aged �18 years with CLI who had a hospitalization or ED visit between 2 January 2021 and 23 April 2022 and had been tested for SARS-CoV-2 within the past 14 days.We excluded individuals who received �1 dose of non-mRNA COVID-19 vaccine.During our sample period, the BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna) vaccines comprised the vast majority (96%) of doses administered in SC.Similar to previous work, we also excluded individuals who tested positive for SARS-CoV-2 >14 days or who received a first dose of mRNA vaccine <14 days or a third dose <7 days before the index encounter [9].

Exposure
The exposure of interest was the receipt of one or more doses of mRNA COVID-19 vaccine.Patients were considered partially vaccinated if they had received one dose of mRNA vaccine �14 days or a second dose <14 days before the index encounter and vaccinated if they had received a second dose of mRNA vaccine �14 days before the index encounter.Fully vaccinated patients who received an additional (booster) dose �7 days before the index encounter were considered to have three-dose protection.Patients who had not yet received any doses of mRNA vaccine as of the index encounter date were considered unvaccinated.
To examine waning effectiveness, we further categorized vaccinated patients by time since vaccination (i.e., number of days between the last vaccine dose and the index encounter).For this purpose, we defined 60-day intervals ranging from �60 days since vaccination to >240 days or >120 days for the second and third dose, respectively.The shorter follow-up period for three-dose protection reflects the fact that, for most groups, a third dose of mRNA COVID-19 vaccine was not recommended until fall 2021.

Outcome of interest and measurement
The primary outcome of interest was a positive or negative SARS-CoV-2 test result for an inpatient or ED patient with CLI.We considered results from tests performed within 14 days of the index encounter.Patients with laboratory-confirmed SARS-CoV-2 infection were classified as cases.Patients with negative SARS-CoV-2 test results were classified as controls.Cases could be included once in the inpatient or ED sample.Where cases had more than one encounter in the inpatient or ED setting, the index date was assigned using the encounter closest to the SARS-CoV-2 test.Controls could be included in the inpatient or ED samples multiple times [9].

Covariates
Covariate selection was informed by known risk factors for COVID-19 infection and severe COVID-19 disease and data availability [28][29][30].Demographic characteristics included age group, sex, race/ethnicity, insurance/payer type, and rural residence.Clinical characteristics included immunocompromised status and eight a priori identified underlying medical conditions: asthma, chronic obstructive pulmonary disease (COPD), cardiovascular disease, congestive heart failure, diabetes mellitus, renal failure, chronic liver disease, and neurological disease.Other variables included indicators corresponding to county, predominant variant, vaccine availability, and hospital teaching status.

Statistical analysis
Separate analyses were conducted for inpatients and ED patients.For each setting, we described characteristics of SARS-CoV-2 positive cases and SARS-CoV-2 negative controls and compared them using the χ 2 test for categorical variables and the two-sample t-test or Mann-Whitney U test for continuous variables.We analyzed the association between COVID-19 vaccination status and laboratory-confirmed SARS-CoV-2 infection using multivariable logistic regression models adjusted for potentially confounding demographic and clinical characteristics.Models were further conditioned on county, predominant variant (Omicron or Delta versus pre-Delta period), and vaccine availability (post-booster versus pre-booster period) such that case-control comparisons were made within the same region and period.We defined the pre-Delta, Delta, and Omicron periods using regional COVID-19 tracking data (reported weekly) and a 50% threshold for variant predominance.
Vaccination status was categorized by number of doses (one, two, or three doses of mRNA vaccine versus unvaccinated).Fully vaccinated patients were further categorized by time since last dose (measured in 60-day intervals).For each vaccination status indicator, we exponentiated the regression coefficient to obtain an adjusted odds ratio (AOR).We calculated VE = (1 -AOR) x 100%.In secondary analyses, we stratified our sample according to age (18-44, 45-64 or �65) to assess VE and the extent of waning for subpopulations with different underlying risk.We also conducted separate analyses for the period September 2021 onward, when a third booster dose was recommended for most groups.Analyses were conducted using SAS software, version 9.4 (SAS Institute) and Stata software, version 17.0 (StataCorp).

Characteristics of study population
Our sample included 37,344 hospitalizations and 79,454 ED visits (Table 1).The inpatient sample included 13,806 patients with a positive SARS-CoV-2 test result.Of those, 24.9%, 39.3%, and 35.7% were admitted during the Omicron, Delta, and pre-Delta periods, respectively.The ED sample included 26,271 patients who tested positive for SARS-CoV-2.Of those, 35.1%, 37.8%, and 27.0% were treated during the Omicron, Delta, and pre-Delta periods, respectively.
In the ED sample, 74.1% of patients were unvaccinated, while 18.0% were fully vaccinated with two doses, and 3.3% had received a third dose of mRNA vaccine.The median time between the encounter date and the most recent vaccination 149 days (IQR: 84-220 days) and 62 days (IQR: 34-95 days) for the second and third dose, respectively.Compared to inpatients
In both the inpatient and ED sample, VE waned as time elapsed from the second dose; however, two-dose protection against ED visits waned more sharply.Comparing VE at �60 days and �240 days from the second dose, protection against ED visits decreased by 38.3 percentage points (pp; 95% CI 29.3-51.0pp).Over the same time period, protection against hospital admissions decreased by just 27.5 pp (95% CI, 22.6-32.5 pp).The difference in the rate of waning (17.6 pp; 95% CI 10.1-25.1) is statistically significant.

Discussion
Two doses of mRNA COVID-19 vaccine were initially highly effective against SARS-CoV-2-related hospital admissions (88.3%; 95% CI, 85.8%-90.4%)and ED visits (83.4%; 95% CI, 80.8%-85.6%).Across all age groups, however, VE waned as time elapsed since the second dose.Compared to protection against hospitalization, protection against ED visits waned more sharply.In all cases, a third dose conferred significant gains in protection relative to waning protection after two doses.In the inpatient sample, VE was 83.6% (95% CI, 80.1%-86.4%)�60 days following a third dose.In the ED sample, VE after a third dose was 82.1% (95% CI, 79.0%-84.7%).Over more than 120 days of follow-up, waning of VE after a third dose was relatively limited.
One strength of our study is the use of integrated state-wide clinical and administrative data to generate population-based estimates of mRNA COVID-19 VE.To date, most realworld evaluations of VE against moderate or severe COVID-19 in the US have been based on analyses of electronic medical records from large regional or public health systems [13,14,17] or sentinel populations [9,[18][19][20].Another strength of our study is the application of a testnegative case-control study design.The test-negative design is a widely applied statistical approach for monitoring VE and is thought to minimize selection bias due to differences in health care access and health care-seeking behavior between vaccinated and unvaccinated patients [24].Our models were further conditioned on county, predominant variant, and vaccine availability such that case-control comparisons were made within the same region and period.Finally, we were able to compare protection against severe (versus moderate) outcomes of COVID-19 in the overall sample as well as for different subgroups defined by age.

Limitations
This study has some limitations.While the test-negative design is thought to reduce unmeasured confounding due to health care-seeking behavior, it is observational in nature and, therefore, subject to potential bias.Owing to our sample period, we were unable to assess longer-term protection of the third dose (i.e., beyond 4 or 5 months) or provide estimates of VE for updated bivalent mRNA COVID-19 vaccines (which were not widely available in the U.S. until fall 2022).

Conclusion
The effectiveness of mRNA COVID-19 vaccines waned as time elapsed from the second dose; however, a third dose conferred significant gains in protection.These findings underscore the importance of mRNA vaccine booster doses during the Delta and Omicron periods and highlight the need for modernized monitoring and public health data systems to facilitate ongoing and more timely population-level estimates of COVID-19 VE in the US.
Fig) and ED (S2 Fig) samples (see S1 Table for

Table 1 . Characteristics of adults who received care for COVID-like illness at a SC hospital or emergency department by SARS-CoV-2 test result.
Hispanic Black (46.1% versus 40.3%) or have private insurance coverage (31.8% versus 21.9%) and less likely to be non-Hispanic White (47.4% versus 53.7%) or have Medicare coverage (23.1% versus 30.7%).
with CLI, ED patients were more racially and ethnically diverse, less likely to have Medicare coverage, and more likely be privately insured, uninsured, or have Medicaid coverage.They were also comparatively younger and had fewer underlying medical conditions.Most patients (55.0%) were under 50 years of age.Common underlying conditions included cardiovascular disease (33.4%), diabetes mellitus (13.2%), and COPD (10.9%); 3.1% of ED patients had an immunocompromising condition.SARS-CoV-2 positive cases were more likely than SARS-CoV-2 negative controls to be unvaccinated (80.2% versus 71.1%).They were also comparatively more likely to be non-